Clusters of post-acute COVID-19 symptoms: a latent class analysis across 9 databases and 7 countries (preprint)

Prior evidence has suggested the multisystem symptomatic manifestations of post-acute COVID-19 condition (PCC). Here we conducted a network cluster analysis of 24 WHO proposed symptoms to identify potential latent subclasses of PCC. Individuals with a positive test of or diagnosed with SARS-CoV-2 after 09/2020 and with at least one symptom within ≥ 90 to 365 days following infection were included. Sub-analyses were conducted among people with ≥ 3 different symptoms. Summary characteristics were provided for each cluster. All analyses were conducted separately in 9 databases from 7 countries, including data from primary care, hospitals, national health claims and national health registries, allowing to validate clusters across the different healthcare settings. 787,078 persons with PCC were included. Single-symptom clusters were common across all databases, particularly for joint pain, anxiety, depression and allergy. Complex clusters included anxiety-depression and abdominal-gastrointestinal symptoms. Substantial heterogeneity within and between PCC clusters was seen across healthcare settings. Current definitions of PCC should be critically reviewed to reflect this variety in clinical presentation.

A nationwide study of 331 rare diseases among 58 million individuals: prevalence, demographics, and COVID-19 outcomes (preprint)

Background: The Global Burden of Disease study has provided key evidence to inform clinicians, researchers, and policy makers across common diseases, but no similar effort with single study design exists for hundreds of rare diseases. Consequently, many rare conditions lack population-level evidence including prevalence and clinical vulnerability. This has led to the absence of evidence-based care for rare diseases, prominently in the COVID-19 pandemic. Method: This study used electronic health records (EHRs) of more than 58 million people in England, linking nine National Health Service datasets spanning healthcare settings for people alive on Jan 23, 2020. Starting with all rare diseases listed in Orphanet, we quality assured and filtered down to analyse 331 conditions with ICD-10 or SNOMED-CT mappings clinically validated in our dataset. We report 1) population prevalence, clinical and demographic details of rare diseases, and 2) investigate differences in mortality with SARs-CoV-2. Findings: Among 58,162,316 individuals, we identified 894,396 with at least one rare disease. Prevalence data in Orphanet originates from various sources with varying degrees of precision. Here we present reproducible age and gender-adjusted estimates for all 331 rare diseases, including first estimates for 186 (56.2%) without any reported prevalence estimate in Orphanet. We identified 49 rare diseases significantly more frequent in females and 62 in males. Similarly we identified 47 rare diseases more frequent in Asian as compared to White ethnicity and 22 with higher Black to white ratios as compared to similar ratios in population controls. 37 rare diseases were overrepresented in the white population as compared to both Black and Asian ethnicities. In total, 7,965 of 894,396 (0.9%) of rare-disease patients died from COVID-19, as compared to 141,287 of 58,162,316 (0.2%) in the full study population. Eight rare diseases had significantly increased risks for COVID-19-related mortality in fully vaccinated individuals, with bullous pemphigoid (8.07[3.01-21.62]) being worst affected. Interpretation: Our study highlights that National-scale EHRs provide a unique resource to estimate detailed prevalence, clinical and demographic data for rare diseases. Using COVID-19-related mortality analysis, we showed the power of large-scale EHRs in providing insights to inform public health decision-making for these often neglected patient populations.

Towards mitigating health inequity via machine learning: a nationwide cohort study to develop and validate ethnicity-specific models for prediction of cardiovascular disease risk in COVID-19 patients (preprint)

Background Emerging data-driven technologies in healthcare, such as risk prediction models, hold great promise but also pose challenges regarding potential bias and exacerbation of existing health inequalities, which have been observed across diseases such as cardiovascular disease (CVD) and COVID-19. This study addresses the impact of ethnicity in risk prediction modelling for cardiovascular events following SARS-CoV-2 infection and explores the potential of ethnicity-specific models to mitigate disparities. Methods This retrospective cohort study utilises six linked datasets accessed through National Health Service (NHS) England's Secure Data Environment (SDE) service for England, via the BHF Data Science Centre's CVD-COVID-UK/COVID-IMPACT Consortium. Inclusion criteria were established, and demographic information, risk factors, and ethnicity categories were defined. Four feature selection methods (LASSO, Random Forest, XGBoost, QRISK) were employed and ethnicity-specific prediction models were trained and tested using logistic regression. Discrimination (AUROC) and calibration performance were assessed for different populations and ethnicity groups. Findings Several differences were observed in the models trained on the whole study cohort vs ethnicity-specific groups. At the feature selection stage, ethnicity-specific models yielded different selected features. AUROC discrimination measures showed consistent performance across most ethnicity groups, with QRISK-based models performing relatively poorly. Calibration performance exhibited variation across ethnicity groups and age categories. Ethnicity-specific models demonstrated the potential to enhance calibration performance for certain ethnic groups. Interpretation This research highlights the importance of considering ethnicity in risk prediction modelling to ensure equitable healthcare outcomes. Differences in selected features and asymmetric calibration across ethnicities underscore the necessity of tailored approaches. Ethnicity-specific models offer a pathway to addressing disparities and improving model performance. The study emphasises the role of data-driven technologies in either alleviating or exacerbating existing health inequalities. Keywords Prediction, machine learning, electronic health records, bias, ethnicity.

Digital ethnicity data in population-wide electronic health records in England: a description of completeness, coverage, and granularity of diversity (preprint)

Background The link between ethnicity and healthcare inequity, and the urgency for better data is well-recognised. This study describes ethnicity data in nation-wide electronic health records in England, UK. Methods We conducted a retrospective cohort study using de-identified person-level records for the England population available in the National Health Service (NHS) Digital trusted research environment. Primary care records (GDPPR) were linked to hospital and national mortality records. We assessed completeness, consistency, and granularity of ethnicity records using all available SNOMED-CT concepts for ethnicity and NHS ethnicity categories. Findings From 61.8 million individuals registered with a primary care practice in England, 51.5 (83.3%) had at least one ethnicity record in GDPPR, increasing to 93·9% when linked with hospital records. Approximately 12·0% had at least two conflicting ethnicity codes in primary care records. Women were more likely to have ethnicity recorded than men. Ethnicity was missing most frequently in individuals from 18 to 39 years old and in the southern regions of England. Individuals with an ethnicity record had more comorbidities recorded than those without. Of 489 SNOMED-CT ethnicity concepts available, 255 were used in primary care records. Discrepancies between SNOMED-CT and NHS ethnicity categories were observed, specifically within “Other-” ethnicity groups. Interpretation More than 250 ethnicity sub-groups may be found in health records for the English population, although commonly categorised into “White”, “Black”, “Asian”, “Mixed”, and “Other”. One in ten individuals do not have ethnicity information recorded in primary care or hospital records. SNOMED-CT codes represent more diversity in ethnicity groups than the NHS ethnicity classification. Improved recording of self-reported ethnicity at first point-of-care and consistency in ethnicity classification across healthcare settings can potentially improve the accuracy of ethnicity in research and ultimately care for all ethnicities. Funding British Heart Foundation Data Science Centre led by Health Data Research UK. Research in context Evidence before this study Ethnicity has been highlighted as a significant factor in the disproportionate impact of SARS-CoV-2 infection and mortality. Better knowledge of ethnicity data recorded in real clinical practice is required to improve health research and ultimately healthcare. We searched PubMed from database inception to 14 th July 2022 for publications using the search terms “ethnicity” and “electronic health records” or “EHR,” without language restrictions. 228 publications in 2019, before the COVID-19 pandemic, and 304 publications between 2020 and 2022 were identified. However, none of these publications used or reported any of over 400 available SNOMED-CT concepts for ethnicity to account for more granularity and diversity than captured by traditional high-level classification limited to 5 to 9 ethnicity groups. Added value of this study We provide a comprehensive study of the largest collection of ethnicity records from a national-level electronic health records trusted research environment, exploring completeness, consistency, and granularity. This work can serve as a data resource profile of ethnicity from routinely-collected EHR in England. Implications of all the available evidence To achieve equity in healthcare, we need to understand the differences between individuals, as well as the influence of ethnicity both on health status and on health interventions, including variation in the behaviour of tests and therapies. Thus, there is a need for measurements, thresholds, and risk estimates to be tailored to different ethnic groups. This study presents the different medical concepts describing ethnicity in routinely collected data that are readily available to researchers and highlights key elements for improving their accuracy in research. We aim to encourage researchers to use more granular ethnicity than the than typical approaches which aggregate ethnicity into a limited number of categories, failing to reflect the diversity of underlying populations. Accurate ethnicity data will lead to a better understanding of individual diversity, which will help to address disparities and influence policy recommendations that can translate into better, fairer health for all.

Use of machine learning for comparing disease risk scores and propensity scores under complex confounding and large sample size scenarios: a simulation study (preprint)

ABSTRACT Background The surge of treatments for COVID-19 in the ongoing pandemic presents an exemplar scenario with low prevalence of a given treatment and high outcome risk. Motivated by that, we conducted a simulation study for treatment effect estimation in such scenarios. We compared the performance of two methods for addressing confounding during the process of estimating treatment effects, namely disease risk scores (DRS) and propensity scores (PS) using different machine learning algorithms. Methods Monte Carlo simulated data with 25 different scenarios of treatment prevalence, outcome risk, data complexity, and sample size were created. PS and DRS matching with 1: 1 ratio were applied with logistic regression with least absolute shrinkage and selection operator (LASSO) regularization, multilayer perceptron (MLP), and eXtreme Gradient Boosting (XgBoost). Estimation performance was evaluated using relative bias and corresponding confidence intervals. Results Bias in treatment effect estimation increased with decreasing treatment prevalence regardless of matching method. DRS resulted in lower bias compared to PS when treatment prevalence was less than 10%, under strong confounding and nonlinear nonadditive data setting. However, DRS did not outperform PS under linear data setting and small sample size, even when the treatment prevalence was less than 10%. PS had a comparable or lower bias to DRS when treatment prevalence was common or high (10% - 50%). All three machine learning methods had similar performance, with LASSO and XgBoost yielding the lowest bias in some scenarios. Decreasing sample size or adding nonlinearity and non-additivity in data improved the performance of both PS and DRS. Conclusions Under strong confounding with large sample size DRS reduced bias compared to PS in scenarios with low treatment prevalence (less than 10%), whilst PS was preferable for the study of treatments with prevalence greater than 10%, regardless of the outcome prevalence. Key Messages When handling nonlinear nonadditive data with strong confounding, DRS estimated by machine learning methods outperforms PS in scenarios with low treatment prevalence (less than 10%). However, if having linear data and small sample size data with strong confounding, we did not observe DRS outperformed PS even when treatment prevalence was less than 10%. Our results suggested that using PS performed better compared to DRS in tackling strong confounding problems with treatment prevalence greater than 10%. Small sample size increased bias for both DRS and PS methods, and it affected DRS more than PS.

A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data (preprint)

Background and Objective As a response to the ongoing COVID-19 pandemic, several prediction models have been rapidly developed, with the aim of providing evidence-based guidance. However, no COVID-19 prediction model in the existing literature has been found to be reliable. Models are commonly assessed to have a risk of bias, often due to insufficient reporting, use of non-representative data, and lack of large-scale external validation. In this paper, we present the Observational Health Data Sciences and Informatics (OHDSI) analytics pipeline for patient-level prediction as a standardized approach for rapid yet reliable development and validation of prediction models. We demonstrate how our analytics pipeline and open-source software can be used to answer important prediction questions while limiting potential causes of bias (e.g., by validating phenotypes, specifying the target population, performing large-scale external validation and publicly providing all analytical source code). Methods We show step-by-step how to implement the pipeline for the question: ‘In patients hospitalized with COVID-19, what is the risk of death 0 to 30 days after hospitalization’. We develop models using six different machine learning methods in a US claims database containing over 20,000 COVID-19 hospitalizations and externally validate the models using data containing over 45,000 COVID-19 hospitalizations from South Korea, Spain, and the US. Results Our open-source tools enabled us to efficiently go end-to-end from problem design to reliable model development and evaluation. When predicting death in patients hospitalized for COVID-19 adaBoost, random forest, gradient boosting machine, and decision tree yielded similar or lower internal and external validation discrimination performance compared to L1-regularized logistic regression, whereas the MLP neural network consistently resulted in lower discrimination. L1-regularized logistic regression models were well calibrated. Conclusion Our results show that following the OHDSI analytics pipeline for patient-level prediction can enable the rapid development towards reliable prediction models. The OHDSI tools and pipeline are open source and available to researchers around the world.

Unraveling COVID-19: a large-scale characterization of 4.5 million COVID-19 cases using CHARYBDIS (preprint)

Background: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response [1,2]. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) [3] Characterizing Health Associated Risks, and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD.Methods: We conducted a descriptive cohort study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub [4]. Findings: We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts, and are available in an interactive website: https://data.ohdsi.org/Covid19CharacterizationCharybdis/. Interpretation: CHARYBDIS findings provide benchmarks that contribute to our understanding of COVID-19 progression, management and evolution over time. This can enable timely assessment of real-world outcomes of preventative and therapeutic options as they are introduced in clinical practice.